Okay, we'll leave theory and come to something eminently practical. Now called machine learning

has been around for centuries essentially. And whenever things get practical, you accept

that. You expect to see very simple mathematics essentially. We're seeing linear models. So

what we are going to get into now is linear regression and classification. So remember,

we were essentially looking at Boolean functions so far with decision lists and decision trees

and all of those kind of things. So we have a problem where we're trying to classify,

say, restaurant situations into I'll wait or I won't wait. So that can be generalized,

of course, to any kind of a function that has a discrete set of examples, outcomes.

Anything you can do for two, you can do for finitely many, and you can even do it for

every natural number. So that's what we call a classification problem. And if we have a

problem where instead we're really, rather than classifying into different sets of three

images of a discrete set, if we are interested in a continuous or real valued result, then

we're going to call that a regression problem. So we're going to look at linear regression

first and to make things really easy, we're going to look at the simplest possible linear

regression problems, namely univariate linear regression, which means we have functions

of one real value. Argument, that's the interesting bit.

So a univariate function or a unary function is a function with one argument, and we'll

go from real numbers to real numbers. And any linear univariate function, we can write

as something times x plus some factor of that. W0 is what basically gives us the slope here,

and W1 gives us the slope, and W0 just moves it up and down. You all know that. And so

we can actually write W0 and W1 as a vector, which we'll use the, in both fields, W4, and

given any such vector, we'll define hW of x to be that linear function induced by that

vector. So all examples, input, output, just become

points into space. And if we look at, say, here we have house prices in terms of the

size, then you can kind of fit in a curve, a linear model that actually predicts house

prices. Interestingly, a house with zero square feet, which is very easy to build, actually

still costs something in Berkeley. But that's the model, right? There's no example here.

That's just what the model tells us. And finding this curve, which is essentially finding this

vector, we call linear regression. There's a typo. So what do we do? So what we want

to do in linear regression is we want to, again, just like last week, we want to minimize

the loss function. In this case, we want to minimize the squared error loss. How do you

do that? Well, you have a couple of examples, the dots here, those here. And so you compute

the loss of a hypothesis given by a two vector w to be the sum of the squares, square loss

of the errors here, which is this. And then if we just basically expand hw, we get such

a little function. The nice thing about that is that we can just essentially compute this

and then we can just basically minimize. How do you do that? How do you minimize a function

like this? Remember high school? A function has a minimum if the first derivative is zero

and the second one is smaller than zero. Okay? Well, we're not like in high school minimizing

the single error argument function, but we actually have a two argument function. Remember

hw is a function that is fully determined by two real numbers, w0 and w1. So this is

actually a function that given a plane full of arguments gives you the loss. So we have

a function that has two arguments, two real arguments and looks quite differentiable.

So what we can do is we can look at the two partial derivatives. One is by understanding

this thing here as a function of w1 and the second one as understanding this as a function

w2. And you'll probably remember calculus one or something like this saying that has

a minimum if the two partial derivatives are zero. Okay? And with a little bit of work,

you can actually work them out. You can solve those equations and you even have a closed

form. Nice. Linear regression is just calculus one or two, whatever. Computing partial derivatives.

So if we want to find out this thing here, we have a couple of x's and y's. We just have

to compute a little sum and then we know these things. Very simple. Okay? That's how simple